Entropic associative memory for manuscript symbols

Manuscript symbols can be stored, recognized and retrieved from an entropic digital memory that is associative and distributed but yet declarative; memory retrieval is a constructive operation, memory cues to objects not contained in the memory are rejected directly without search, and memory operations can be performed through parallel computations. Manuscript symbols, both letters and numerals, are represented in Associative Memory Registers that have an associated entropy. The memory recognition operation obeys an entropy trade-off between precision and recall, and the entropy level impacts on the quality of the objects recovered through the memory retrieval operation. The present proposal is contrasted in several dimensions with neural networks models of associative memory. We discuss the operational characteristics of the entropic associative memory for retrieving objects with both complete and incomplete information, such as severe occlusions. The experiments reported in this paper add evidence on the potential of this framework for developing practical applications and computational models of natural memory.

experiments are already on their way. More generally, an effective associative memory should be a central functional module of the cognitive architecture of computational agents that need to interact in real time with the world." 4. Compared with the previous publications, what is the advantage of this paper?

Response:
The experiments presented in this paper deal with a much larger number of classes than the previous one: 47 and 36 versus 10 and 5, respectively, with satisfactory results. In addition, a detailed comparison in 10 dimensions with previous work in associative memories developed within the framework of Artificial Neural Networks (ANNs) is presented in Section 5. These include (1) EAM's the declarative format versus the procedural or sub-symbolic format of ANNs; (2) the productive or generalization property of EAM that ANNs' memories lack; (3) the constructive retrieval of EAM versus the photographic retrieval of ANNs approaches; (4) the integrated auto and hetero associativity of EAM; (5) the property of direct rejection of EAM that ANNs approaches lack; (6) the natural parallelism of EAM due to the direct cell to cell and column operations, which does not involve matrix operations; (7) EAM is not a dynamic system and does not use an energy function; (8) the explicit use of the entropy by EAM that ANNs approaches lack; (9) the very large storing capacity of EAM; and (10) the quantitative comparisons of EAM with ANNs; in particular, with associative memory systems using similar corpus, such as dense associative memories (Krotov and Hopfield, Ref. 29).

The format of the references shall be united
Response: All references have been reviewed and included using the Latex BibTex format, using PLOS ONE templates.
6. The following related publications on associative memory shall be added: Response: Our paper is concerned with the performance of a novel model of associative memory that uses a declarative format in which memory register is an abstraction, cues not contained in the memory are rejected directly without search, and memory retrieval is a constructive operation. We assessed our model empirically in terms of the precision, the recall and the accuracy of the memory recognition and the memory retrieval operations in relation to the memory size, the amount remembered data and the entropy, and the tolerance to error; and we offered a comparison with related work addressing similar problems. We could not find any of these concerns in the recommended papers and are unable to appreciate why such references shall be included, and we have not done so.
The answers to reviewer 2 are: 1. On page 4, the amount of data allocated by Training Corpus, Remembered Corpus and Test Corpus is 57%, 30% and 10%, respectively. Please explain in detail why such a proportion is allocated and the advantages of this allocation.

Response:
A new paragraph was added in Section 3, p.7 as follows: "The data allocated to each partition reflects a trade-off between learning and test data, so 90% of the corpus is used for the former and 10% for the latter, according to standard machine learning practices. The balance between training and remembered data considers that a large enough amount of corpus is needed for training the deep neural networks modeling perception and action, but enough data is needed to test the AMRs with different remembering conditions and entropy levels. We also made preliminary experiments with small variations to these amounts, and the present choice constitutes a satisfactory compromise." 2. In Figure 7, when the totality of the remembered corpus is used, the recall is lower and the graphs do not intersect. For the largest registers, the cost of memory resources is twice as high. How to solve these problems in practice.

Response:
The fact that the precision and the recall do not intersect with a given amount of data is not necessarily a problem in itself. Figure 7 shows that grids of 64 and 128 rows perform better for the EMNIST dataset than smaller and larger grids, and that the best performance is achieved using the totality of the remembered data. Hence these sizes are considered for functional AMRs and are tested with different amounts of corpus and entropy levels. The graphs show that the best performance for this dataset is achieved using the totality of the remembered corpus: the results also suggest that registering more information will not impact on the performance significantly. As the performance is very similar in both settings the smallest AMRs size is chosen due to its reduced cost. A related but different question is how to improve the performance of the system as a whole and reduce the memory size for an arbitrary dataset. For this we plan to reinforce the cells of the AMRs whenever they are used in the memory register mechanism, such that columns become probability distributions shaped by the empirical data, with their associate Shannon's entropy. Such learning mechanism should improve the performance and reduce the size of operational AMRs. We leave such investigation for further work.
A new paragraph addressing this latter question has been added in Section 6, p. 22, as follows: "The construction of practical applications requires improving the performance of the system as a whole and reduce the memory size for an arbitrary dataset. In order to address such question, we plan we plan to reinforce the cells of the AMRs whenever they are used in the memory register operation, such that columns become probability distributions shaped by the empirical data, with their associate Shannon's entropy. Such learning mechanism should improve the performance and reduce the size of operational AMRs. We leave such investigation for further work."